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Abstract 

A  proposed  plan  for  complete  implementation  of  the  Picture  Archiving  and 
Communication  System  (PACS)  at  Walter  Reed  Army  Medical  Center  (WRAMC)  was  assessed 
by  (1)  cost  identification  analysis  and  (2)  provider  satisfaction  (employing  an  1 1-item  survey 
instrument).  The  annual  operating  cost  for  year  1  amounts  to  $2,485,924.80  and  ranges  from 
$2,774,724.80  to  $1,812,861.80  for  years  2  through  8.  In  contrast,  the  annual  cost  of  film-based 
radiology  is  $913,665.97.  Survey  participants  (n=l  19)  agreed  highly  that  PACS  provides  rapid 
access  to  radiographic  images  and  that  the  presence  of  multiple  viewing  stations  is  important  in 
their  clinical  practice.  Furthermore,  they  are  confident  in  their  diagnoses  using  digital  images  and 
they  agree  that  the  PACS  improves  the  quality  of  care  delivered.  Overall  satisfaction  with  the 
current  PACS  was  moderate  (M  =  2.17  ±  .87)  based  on  a  five-point  Lickert  scale  where  1.00 
equaled  complete  satisfaction.  Satisfaction  varied  by  department  and  type  of  provider.  Stepwise 
multiple  regression  analysis  revealed  that  the  most  significant  factors  associated  with  overall 
satisfaction  are  (1)  "user  friendliness,"  (2)  the  effect  on  efficiency  of  clinical  practice,  (3) 
reliability,  and  (4)  confidence  in  diagnosis  using  PACS,  which  resulted  in  a  shared  variance, 

R2  =  .557  with  F  (4,  1 14)  =  35.847,  p  <  .0001 .  Recommendations  are  presented  for  expansion  of 
PACS  at  WRAMC.  PACS  offers  distinct  advantages  when  compared  to  conventional  film-based 
radiology.  However,  hardware,  software,  information  system,  and  maintenance  costs  of  a  PACS 
are  high.  Clinicians  and  hospital  administrators  must  determine  if  a  PACS  is  the  best  method, 
consistent  with  their  budget,  to  solve  problems  with  storage  and  distribution  of  medical  image 
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Introduction 

Telemedicine  has  many  definitions,  but  may  be  described  as  a  combination  of 
telecommunications  technology  and  medical  expertise  designed  to  enhance  the  delivery  of 
medical  care  or  medical  education  (Goldberg,  1996).  Use  of  communication  systems  to 
disseminate  medical  expertise  is  not  new.  An  article  describing  the  examination  of  a  child  at 
home  using  interactive  videoconferencing  appeared  in  the  magazine  Radio  News  in  April  1924, 
three  years  before  the  invention  of  the  television.  However,  first  generation  telemedicine  systems 
were  not  deployed  until  the  1960s  and  1970s. 

Today  there  is  renewed  interest  in  telemedicine,  which  may  be  attributed  to  (1)  rapid 
advances  in  telecommunication  and  information  management  technologies  that  are  applicable  to 
the  delivery  of  medical  services  and  (2)  increased  demand  for  equal  access  to  high  quality 
medical  care  at  a  reasonable  cost  (Goldberg,  1996). 

Teleradiology,  the  most  mature  of  telemedicine  “subspecialties,”  has  benefited  from  more 
than  twenty  years  of  research  on  the  application  of  digital  technology  to  the  display  and 
management  of  radiographic  images,  with  particular  emphasis  on  transmission  of  those  images 
both  within  and  between  medical  treatment  facilities  (Goldberg,  1996).  Teleradiology  systems 
have  several  applications.  This  technology  may  be  used  for  the  purpose  of  primary  readings 
(without  review  of  original  radiographs).  These  systems  may  also  be  used  for  consultation  or 
second  opinions  in  difficult  cases  or  to  supplement  coverage  after  normal  working  hours  or 
during  planned  or  unexpected  absences  in  a  radiology  practice. 

During  the  past  two  decades,  radiology  departments  have  experienced  a  progressive  shift 
from  capturing  an  image  on  a  photographic  plate  to  capturing  it  with  radiation  detectors  whose 
output  signals  can  be  digitized  (Hynes,  Stevenson  &  Nahmias,  1997).  Digital  radiology  has 
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distinct  advantages.  All  data  are  preserved,  transported,  sorted,  read,  stored,  and  retrieved 
electronically  and  expeditiously.  Additionally,  image  modification  is  possible.  Whereas  images 
on  conventional  films  are  fixed  (the  only  modification  possible  is  to  use  a  brighter  light),  the 
contrast  of  digital  images  may  be  altered  to  improve  viewing  quality.  In  contrast  to  digital 
images,  conventional  radiographic  film  is  expensive,  can  be  used  in  just  one  location  at  a  time, 
requires  a  great  deal  of  storage  space,  may  be  over-  or  under-exposed,  and  often  can  be  lost  or 
misplaced  (Hynes  et  al.,  1997). 

The  term  “Picture  Archiving  and  Communication  System”  (PACS),  also  known  as  image 
management  and  communication  system  (Tucker,  Barnes  &  Koehler,  1995a),  refers  to  networks 
of  digital  image  modalities,  work  and  viewing  stations,  and  storage  units  that  are  connected  to 
each  other  by  image  data  communication  structures  and  are  controlled  by  appropriate  image  and 
data  management  (Meyer-Ebrecht,  1994).  The  implementation  of  PACS  is  an  attempt  to  solve  a 
major  problem  of  diagnostic  medicine:  to  store  a  huge  archive  of  radiographs  in  such  a  manner 
that  access  to  the  images  becomes  as  rapid  and  efficient  as  access  to  data  and  text  in  office 
computer  systems. 

For  years,  photographic  film  has  been  considered  the  “gold  standard”  for  presentation, 
transport,  and  storage  of  image  information.  However,  the  sheer  number  of  films  produced  (in 
large  hospitals  about  1,000  images  are  produced  daily)  creates  several  problems  (Meyer-Ebrecht, 
1994).  A  considerable  amount  of  effort  is  required  to  make  films  available  for  daily  practice  and 
to  archive  them  so  they  are  available  within  a  reasonable  amount  of  time.  An  archived  image 
must  be  available,  within  an  acceptable  period  of  delay,  at  any  given  time  for  a  period  of  at  least 
five  years.  Radiographic  film  archives  of  several  million  images  can  occupy  a  considerable 
amount  of  space  and  the  large  number  of  films  may  preclude  timely  access  to  urgently  required 
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images.  In  addition,  films  cannot  be  simultaneously  available  at  different  places  for  consultation. 
Substitution  of  digital  images  for  films  solves  many  problems  of  storage  and  retrieval  of 
radiographic  images.  All  diagnostic  workstations,  where  images  are  produced  and  evaluated,  are 
connected  with  the  digital  archive  by  way  of  digital  networks.  Physical  retrieval  and 
transportation  of  radiographic  films  (requiring  considerable  manpower)  becomes  unnecessary; 
digital  networks  permit  timely  access  to  radiographic  images  simultaneously  at  multiple 
locations,  regardless  of  whether  they  have  just  been  generated  or  have  been  archived  for  many 
years.  (Meyer-Ebrecht,  1994). 

A  PACS  consists  of  four  components:  (1)  image  acquisition  devices;  (2)  a  network  for 
transmission  and  distribution  of  images;  (3)  workstations  for  image  display  and  interpretation; 
and  (4)  image  storage  and  retrieval  systems;  (Nissen-Meyer,  Fink,  Pleier  &  Becker,  1996;  Hynes 
et  al.,  1997). 

An  image  acquisition  device  serves  as  the  point  of  entry  for  images  into  the  PACS 
(Tucker  et  al.,  1995a).  In  computed  radiography  (CR),  which  was  introduced  by  Fugi  Photo  Film 
Co.,  Ltd.  in  1983,  digital  radiographic  images  are  produced  directly  (Sonada,  Takano,  Miyahara 
&  Kato,  1983)  using  a  storage  phosphor  system  (SPS).  SPS  employs  a  re-usable  phosphor-coated 
film,  which  is  sensitive  to  x-radiation.  CR  is  compatible  with  most  conventional  x-ray  imaging 
equipment  and  has  a  wide  exposure  latitude,  producing  images  with  consistent  density  over  a 
wide  range  of  exposure  levels,  thereby  eliminating  over-  and  under-exposure  problems 
(Miyahara,  1987).  Disadvantages  of  CR  include  increased  “scatter”  sensitivity  (Tucker,  Souto  & 
Barnes  1993),  limited  spatial  resolution  (Kato,  1994)  and  increased  cost  (Tucker  et  al.,  1995a). 

Image  distribution  refers  to  movement  of  a  radiographic  image  from  one  location  to 
another,  i.e.  from  an  acquisition  device  or  storage  device  to  one  or  more  clinical  workstations 
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(Tucker  et  al.,  1995a).  Image  distribution  may  be  accomplished  using  a  Hospital  Information 
System  (HIS),  Radiology  Information  System  (RIS)  or  Local  Area  Network  (LAN).  Optimizing 
image  distribution  is  a  tradeoff  between  expense  and  bandwidth.  Increased  bandwidth  permits 
more  rapid  transmission  of  images  and  enhances  network  capacity  (Goldberg,  1996).  Important 
factors  to  be  considered  include  the  anticipated  number  of  images  to  be  transmitted,  the  case  mix 
(which  determines  the  average  size  of  individual  files),  the  required  turnaround  time,  and  periods 
of  peak  activity.  Consideration  of  peak  activity  is  important  to  avoid  “bottlenecks”  that  may 
result  from  periods  of  increased  activity  during  one  or  more  portions  of  the  day. 

Critical  to  the  success  of  PACS  is  the  physician’s  ability  to  derive  useful  information 
from  digital  radiographs,  facilitating  diagnosis  and  formation  of  an  appropriate  treatment  plan 
(Tucker  et  al.,  1995a).  Factors  to  be  considered  include  ease  of  use,  sufficient  number  and  quality 
(resolution)  of  monitors,  and  the  strategic  location  and  reliability  of  workstations  that  serve  a 
wide  spectrum  of  users.  Distribution  of  clinical  workstations  is  based  on  peak  clinic  activity,  the 
average  time  required  for  image  review,  the  geographic  layout  of  the  clinic  and  the  location  of 
conference  rooms  (Smith,  Smith  &  Sauls,  1992;  Leckie,  Goeringer  &  Smith,  1993).  The 
American  College  of  Radiology  (ACR)  developed  standards  for  teleradiology  in  1994  (Goldberg, 
1996).  For  most  conventional  images,  the  ACR  recommends  a  resolution  of  2K  X  2K  X  12  bits 
for  image  acquisition  and  display.  Use  of  dual  monitors  permits  easy  side-by-side  comparison  of 
current  and  previously  obtained  images.  Interactive  controls  offer  the  opportunity  to  adjust 
contrast,  brightness,  magnification,  and  position  of  images  and  to  perform  accurate 
measurements  (Gillespy  &  Rowberg,  1994;  Goldberg,  1996).  To  simplify  the  image  selection 
process,  a  limited  view  of  the  image  database  should  be  provided;  it  should  be  targeted  to  the 
primary  clinical  procedure  accomplished  where  the  workstation  is  used.  The  elapsed  time 
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between  selection  of  an  image  and  the  appearance  of  that  image  on  the  display  should  be 
minimal  (ideally  less  than  five  seconds).  Retrieval  of  images  on  an  ad  hoc  basis  during  the  day 
could  potentially  overwhelm  the  system  if  multiple  users  request  archived  images  simultaneously 
(Smith,  Smith,  Bender,  Carter,  Kim,  Cawthon,  Leckie,  Weiser,  Romlein  &  Goeringer,  1995).  To 
prevent  such  an  occurrence,  an  “intelligent  prefetch”  may  be  accomplished:  a  clinic  appointment 
list  may  be  provided  to  the  archive  system  by  way  of  the  HIS  or  RIS,  so  that  images  may  be 
dearchived  during  the  preceding  night  and  transmitted  to  a  short-term  storage  system,  enabling 
timely  retrieval  the  next  day.  Finally,  radiographic  interpretation  (Tucker  et  al.,  1995a)  and 
image  annotation,  including  the  patient’s  demographic  and  examination  information  (Goldberg, 
1996)  should  be  included  and  viewed  with  all  images,  in  accordance  with  ACR  guidelines. 

Long-term  storage  and  retrieval  of  digital  images  presents  a  significant  challenge  in 
PACS  due  to  the  sizeable  amount  of  information  involved.  One  digitized  page  of  text  requires  a 
data  set  of  thousands  of  bytes,  whereas  a  digital  radiograph  requires  a  data  set  of  millions  of 
bytes  (Meyer-Ebrecht,  1994).  A  typical  chest  x-ray,  for  example,  requires  at  least  2000  X  2000 
pixels  and  necessitates  the  use  of  at  least  10  bits  to  record  the  optical  density  of  each  pixel. 
Mammograms  and  bone  radiographs,  which  require  greater  sharpness  of  detail,  may  require  a 
data  set  of  not  less  than  5  mb  (Seely,  Fisher,  Stempski,  Borgstrom,  Bjelland,  &  Capp,  1987). 
Since  the  data  must  be  stored  for  at  least  five  years  and  indefinitely  for  mammograms  (Smith  et 
ah,  1995),  and  since  as  many  as  2  million  images  may  be  produced  annually  in  a  typical  hospital 
(Meyer-Ebrecht,  1994),  the  storage  system  must  accommodate  10  tb  of  data.  Storage  and 
retrieval  solutions  for  these  giant  data  sets  must  be  technically  and  financially  practical  for  the 
treatment  facility  and  must  include  consideration  of  the  institution’s  short-  and  long-term  PACS 
plans.  Images  may  be  archived  in  a  database  associated  with  the  individual  workstation, 
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permitting  immediate  access  for  the  duration  of  the  patient’s  stay  at  the  hospital.  Upon  discharge 
of  the  patient,  long-term  management  and  storage  of  images  may  be  accomplished  by  a  separate 
archive  system  either  within  the  department  of  radiology  or  at  a  centralized  storage  facility.  With 
the  introduction  of  laser-scanned  optical  storage  disks  in  the  1970’s,  storage  of  several  gb  of  data 
became  possible.  Today,  double-sided  optical  disks  are  stored  in  “juke-box”-like  disk  changers, 
which  occupy  a  fraction  of  the  space  required  to  store  conventional  radiographic  films  (Meyer- 
Ebrecht,  1994). 

Important  PACS  design  concepts  include  standardization,  open  architecture,  connectivity 
and  reliability  (Huang  &  Taira,  1992).  To  minimize  the  need  to  develop  its  own  software  and  to 
maximize  the  portability  of  the  PACS  to  other  computer  platforms,  a  treatment  facility  should 
incorporate  as  many  industry  standards  as  possible.  Further,  an  open  network  design  architecture 
permits  a  standardized  method  for  data  exchange  between  diverse  systems.  In  this  era  of  rapid 
advances  in  computer  technology,  a  closed  architecture  would  hinder  efforts  to  upgrade  the 
system.  Reliability  and  redundancy  are  important  considerations  due  to  the  fact  that  a  PACS  has 
a  myriad  of  components  (thus,  the  probability  of  a  component  failing  is  high).  Since  PACS 
manages  and  displays  crucial  patient  information,  it  cannot  be  inoperable  for  extended  periods  of 
time.  Finally,  system  security  is  a  major  concern  due  to  medico-legal  issues  regarding  patient 
confidentiality.  Most  PACS  have  mechanisms  to  control  access  to  images  and  patients’ 
demographic  and  examination  information.  These  include  account  control  (use  of  accounts  and 
passwords)  and  privilege  control  (granting  or  revoking  access  to  specific  images  or  data). 

Conditions  which  Prompted  the  Study 

The  potential  benefits  of  PACS  include  both  quantitative  and  qualitative  improvements  in 
the  practice  of  radiology  (Siegel  &  Brown,  1994).  PACS  could  potentially  replace  traditional 
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film-based  techniques,  leading  to  a  “filmless”  radiology  department,  where  users  have  rapid 
access  to  radiological  images  both  within  a  medical  treatment  facility  and  from  remote  locations 
(teleradiology).  Realization  of  this  scenario,  however,  is  dependent  upon  several  factors, 
including  continued  improvements  in  the  performance  of  system  hardware  and  software, 
decreased  system  costs,  and  acceptance  by  medical  personnel  (Siegel  &  Brown,  1994).  Some 
radiologists  view  PACS  as  an  exciting  innovation  that  will  enhance  and  expand  the  practice  of 
radiology;  others  view  it  as  an  expensive  technology  with  marginal  benefits  (Dure-Smith  & 
Fymat,  1997). 

Madigan  Army  Medical  Center  (MAMC)  was  the  first  U.S.  Army  site  for  installation  of  a 
PACS  under  the  Medical  Diagnostic  Imaging  Support  (MDIS)  system  contract  for  the  U.S. 
Department  of  Defense  (Smith  et  al.,  1995).  Clinical  operations  began  in  March  1992,  with  the 
goal  of  achieving  90  to  95  percent  filmless  operations  upon  complete  implementation.  Smith  et 
al.  (1995)  report  that  system  effectiveness  and  performance  have  been  good  with  regard  to 
workload  throughput  and  reliability.  Clinical  physicians  readily  accept  this  new  technology; 
radiologists,  however,  are  less  enthusiastic  about  the  use  of  clinical  workstations,  presumably 
because  MAMC  is  still  in  the  process  of  implementing  the  full  complement  of  hardware  and 
software  features. 

Planning  for  a  PACS  at  Walter  Reed  Army  Medical  Center  (WRAMC)  began  in  January, 
1994.  Implementation  of  the  first  phase  of  the  system  occurred  in  February,  1997  at  a  cost  of 
approximately  $4  million.  The  current  PACS  hardware  includes  three  diagnostic  workstations 
(“4A’s”)  for  use  by  radiologists,  ten  clinical  review  stations  (“2C’s”)  for  use  by  other  physicians, 
a  Fugi  AC-3CS  processor,  a  Computer  Radiology  Acquisition  Workstation  (CRAW),  an  Interim 
Storage  Unit  (ISU),  and  an  Optical  Disk  Jukebox  (ODJ)  long-term  storage  unit.  Future  plans  call 
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for  installation  of  review  stations  in  each  hospital  clinic.  Digital  images  are  currently  stored  for 
outpatient  clinics,  the  intensive  care  unit  (ICU),  and  the  emergency  department  (ED).  Future 
upgrades  to  the  PACS  will  permit  storage  of  images  for  Computed  Tomography  (CT),  Magnetic 
Resonance  Imaging  (MRI),  Ultrasound,  Nuclear  Medicine,  and  Orthopaedics. 

The  process  of  obtaining  a  digital  radiograph  is  as  follows:  a  physician  orders  a 
radiograph  using  the  Composite  Health  Care  System  (CHCS),  an  element  of  the  hospital 
information  system  (HIS),  which  records  and  transmits  patient  information  and  data.  Upon 
arrival  at  the  radiology  department,  the  patient  is  escorted  to  an  examination  room  and  the 
radiology  examination  is  performed  using  a  reusable,  phosphorous-coated  imaging  plate  in  lieu 
of  conventional  x-ray  film.  Patient  demographic  information  is  transferred  by  way  of  the  HIS  to  a 
patient  terminal  (PT),  shown  in  Figure  1 .  The  information  is  entered  into  a  Medical  Digital 
Imaging  Support  system  (MDIS,  General  Electric  Corporation)  and  is  then  transmitted  to  an 
identification  terminal  (IDT).  Here,  patient  demographic  information  is  checked  for  accuracy. 

The  exposed  imaging  plate  is  placed  into  the  Fugi  AC-3CS  processor  (equipped  with  a  “stacker” 
to  accommodate  four  plates  simultaneously)  which  pre-scans  and  processes  the  radiographic 
image  (see  Figure  2).  Patient  identification  data  and  the  radiographic  image  are  transferred  to  the 
CRAW  (see  Figure  1),  where  a  quality  control  senior  radiology  technician  verifies  the  patient 
data,  reviews  the  image,  makes  appropriate  adjustments  to  enhance  contrast  and  position,  and 
sends  the  data  and  image  electronically  to  the  ISU.  Here,  they  are  available  for  review  by  a 
radiologist  at  a  4 A  workstation,  as  shown  in  Figure  3.  The  image  is  then  transferred 
electronically  to  the  OD J  for  long-term  storage  and  future  retrieval  at  a  2C  viewing  station  (see 
Figure  4).  The  ODJ  contains  100  removable  optical  disks  (twelve-inch  platters).  Each  optical 
disk  can  store  the  equivalent  of  about  100,000  chest  radiographs. 
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The  Chief,  Department  of  Radiology,  has  expressed  interest  in  an  evaluation  of  the  costs 
and  benefits  of  the  PACS  at  WRAMC.  In  deciding  to  implement  the  system  in  February  1997, 
management  followed  an  industry  trend  that  was  supported  by  favorable  results  of  the  system  at 
MAMC.  It  is  now  appropriate  to  evaluate  the  efficacy  of  the  PACS  compared  to  a  conventional 
film-based  system  for  radiology.  Hospital  executives  request  a  report  on  the  return  for  the 
sizeable  investment  in  PACS. 

Research  Questions 

The  primary  research  questions  addressed  in  this  study  are:  (a)  How  does  the  cost  of  the 
PACS  compare  with  the  cost  of  conventional  film-based  radiology  at  WRAMC?  (b)  What  are  the 
benefits  (as  perceived  by  clinicians)  of  the  PACS  at  WRAMC?  (c)  Are  WRAMC  physicians 
satisfied  with  the  PACS?  (d)  What  recommendations  can  be  made  for  future  courses  of  action 
regarding  full  implementation  of  the  PACS  at  WRAMC? 

Literature  Review 

In  1993,  the  United  States  spent  more  than  $900  billion  (more  than  14  percent  of  our 
gross  national  product  (GNP)  on  health  care  (Feldstein,  1994),  and  medical  expenditures  are 
rising  at  a  rate  of  1 1  to  12  percent  each  year  (Letsch,  1993).  If  this  rate  continues,  health  care 
spending  could  reach  $1.6  trillion  (18  percent  of  our  GNP)  by  the  turn  of  the  century  (Feldstein, 
1994). 

Advances  in  medical  science  and  health  care  technology  since  the  end  of  World  War  II 
have  fueled  growth  in  health  care  costs  (Williams  &  Torrens,  1993).  New  methods  of  diagnosis 
and  treatment  have  stimulated  demand  for  medical  treatment  and  providers  of  health  care  (both 
hospitals  and  physicians)  have  responded  to  the  increased  demand  for  care  (Feldstein,  1994). 
Until  recently,  there  were  few  incentives  for  hospitals  to  contain  costs;  health  care  facilities 
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concentrated  on  generating  income  and  could  be  classified  as  “revenue  centers.”  With  the 
implementation  of  a  new  Medicare  hospital  payment  system  in  1983,  hospitals  were  no  longer 
reimbursed  according  to  their  expenses.  Instead,  fixed  costs  were  established  for  each  diagnostic 
admission,  referred  to  as  diagnostic  related  groups  (DRG).  Hospitals,  therefore,  began  to  focus 
on  limiting  expenses  and  became  “cost  centers.” 

Diffusion  refers  to  the  spread  of  a  technology  into  society  once  it  is  developed  (Williams 
&  Torrens,  1993).  In  essence,  it  is  the  life  cycle  of  technology  including  entry,  adoption, 
widespread  use,  and  final  obsolescence  of  an  invention.  The  public,  of  course,  is  concerned  about 
access  to  new  and  beneficial  procedures  and  devices.  In  contrast,  payers  (including  private 
insurance  companies  and  federal  and  state  governments),  are  more  concerned  with  assessing  the 
costs,  benefits  and  efficacy  of  new  technology  before  it  is  adopted  (Williams  &  Torrens,  1993). 

Today,  health  care  facilities  are  confronted  with  a  myriad  of  new  medical  technologies 
that  are  presumed  to  enhance  therapy,  diagnosis,  care  or  infrastructure  (Van  Gennip  &  Gremy, 
1993).  Since  financial  resources  are  limited,  hospitals  cannot  adopt  all  of  these  innovations. 
Therefore,  health  care  decision-makers  must  consider  costs  and  benefits  in  the  selection  of  new 
technology. 

Technology  assessment  provides  insight  into  the  costs  and  effectiveness  of  technology 
(Van  Gennip  &  Gremy,  1993).  Although  many  methods  of  technology  assessment  focus 
primarily  on  safety  and  efficacy,  cost-effectiveness  analysis  (CEA)  has  become  increasingly 
popular,  since  it  is  apparent  that  when  a  given  technology  is  adopted,  there  are  economic 
constraints  that  require  trade-offs  (Warner  &  Luce,  1982). 

Warner  and  Luce  (1982)  describe  a  framework  for  conducting  a  CEA:  (a)  define  the 
problem  and  objectives;  (b)  identify  alternatives  to  address  the  problem;  (c)  identify,  measure 
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and  value  costs  and  benefits;  (d)  compare  alternatives;  and  (e)  present  the  findings  to 
stakeholders.  Technology  assessments  that  follow  this  framework  are  problem-oriented,  rather 
than  technology-oriented.  Competing  technologies  are  evaluated  based  on  their  appropriateness 
in  solving  the  defined  problem  (Van  Gennip  &  Gremy,  1993). 

Because  of  the  volume  of  data  managed,  the  required  quality  of  images  displayed,  the 
speed  of  transmission  and  the  necessity  of  high-tech  equipment,  PACS  require  a  substantial 
financial  sacrifice  (Van  Gennip  &  Bakker,  1993).  Hence,  it  behooves  hospital  administrators  to 
weigh  the  costs  and  benefits  before  deciding  to  invest  in  this  technology.  The  purpose  of 
technology  assessment  of  PACS  is  to  establish  the  need  for  the  technology,  provide  a  measure  of 
its  function,  and  identify  the  costs  and  benefits  associated  with  implementation  of  the  system 
(Crowe,  1992).  Technology  assessment,  therefore,  seeks  “to  apply  economics  to  cost  and 
rationality  to  choice”  (Wells,  Garrett  &  Jackson,  1991,  p.93). 

CEA  is  widely  used  as  a  tool  for  technology  assessment  (van  Gennip  &  Bakker,  1993).  It 
implies  that  the  cost  of  an  innovation  may  be  compared  to  its  effectiveness  and  that  the  latter  is 
not  (per  se)  expressed  in  monetary  units  (Warner  &  Luce,  1982).  Most  CEA  studies  define 
effectiveness  in  terms  of  health  care  outcomes,  such  as  dollars  per  year  of  life  saved.  However, 
Donobedian  (1982)  defines  effectiveness  much  more  broadly,  including  benefits  in  the  process. 

The  concept  of  the  “technological  imperative”  assumes  that  simply  because  a  technology 
exists,  it  will  be  adopted  and  used.  Crowe  (1992)  asserts  that  this  is  not  necessarily  true,  and  that 
the  history  of  technology  is  littered  with  brilliant  ideas  that,  for  one  reason  or  another,  were  never 
adopted.  Crowe  (1992)  examined  the  features  of  a  technology  which  will  lead  to  its  widespread 
introduction  and  acceptance:  (a)  it  must  fulfill  a  profound  need  of  a  large  group  of  potential 
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users;  (b)  it  must  perform  a  function  better  than  existing  technology;  and  (c)  it  must  be  available 
at  the  same  or  lower  cost  than  existing  systems. 

Technology  assessment  enables  hospital  administrators  to  make  informed  decisions 
regarding  allocation  of  scarce  resources.  Drummond,  Stoddard  and  Torrance  (1987)  emphasized 
the  need  for  assessment  and  the  concept  of  “opportunity  costs.”  In  an  arena  of  constrained 
resources,  those  devoted  to  one  technology  are  denied  to  another.  Technology  assessment 
permits  comparison  of  competing  technologies  on  a  uniform  basis  with  regard  to  need, 
functionality,  costs,  and  benefits  (Crowe,  1992).  To  be  accepted,  a  technology  must  be  effective, 
economic,  appropriate  and  needed  (Wells  et  al.,  1991). 

Determining  the  need  for  a  technology  involves  consideration  of  subjective  factors 
(Crowe,  1992).  Most  important  of  these  is  perceived  or  demonstrated  dissatisfaction  with 
alternative  systems.  (With  regard  to  PACS,  the  only  alternative  is  use  of  conventional 
radiographic  film.)  The  level  of  dissatisfaction  must  be  such  that  employees  are  willing  to  endure 
the  inconvenience  that  accompanies  introduction  of  new  technology  and  the  associated  “learning 
curve.” 

Functional  assessment  is  a  method  of  determining  if  a  new  technology  is  capable  of 
performing  a  task(s)  better  (e.g.  faster,  more  efficiently,  and  more  cost-effectively)  than  an 
alternative  system  (Crowe,  1992).  Performance  evaluation  may  be  complicated  by  the 
introduction  of  improved  hardware  and  new  techniques  (the  “moving  target”  phenomenon). 
Assessment  is,  therefore,  most  challenging  in  the  early  days  of  new  technology,  when  the  tempo 
for  technical  change  is  rapid  and  there  is  compelling  pressure  for  acquisition  because  of 
“apparently  self-evident  advantages”  (Wells  et  al.,  1991,  p.  94). 
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Hospital  administrators  demand  performance  of  cost-effectiveness  analyses  before 
making  an  investment  in  PACS  (Crowe,  1992).  Some  studies  (Andriessen,  Ter  Haar  Romeny, 
Binkhuysen  &  Van  der  Horst-Bruinsma,  1989;  Warburton,  Fisher  &  Nosil,  1990;  Van  Gennip, 
Bakker  &  Greberman,  1992)  have  determined  that  the  cost  of  PACS  is  higher  than  conventional 
film  systems.  However,  costs  must  be  examined  in  relation  to  expected  benefits  of  PACS 
(Crowe,  1992).  Benefits  peculiar  to  PACS  are  presented  in  Table  1.  Crowe  (1992,  p.l  85)  argued 
that  “far  too  much  attention  has  been  paid  to  the  itemization  of  costs  of  PACS  systems,  without 
sufficient  attention  being  paid  to  the  discrimination  of  benefits.”  Cost-benefit  assessment  of 
PACS,  therefore,  involves  far  more  than  financial  considerations  alone  (Strickland,  1996);  it 
should  include  not  only  an  examination  of  costs,  but  also  effectiveness,  economy,  and 
appropriateness.  The  net  result  should  be  an  increase  in  the  health  of  the  patient  population 
served  (Wells  et  al.,  1991). 

PACS  may  offer  a  medical  treatment  facility  the  promise  of  lower  operating  costs, 
increased  efficiency,  and  improved  quality  of  care  (Langlotz,  Even-Shoshan,  Seshadri, 

Brickman,  Kishore,  Kundel  &  Schwartz,  1995).  As  the  technical  quality  of  PACS  improves  and 
large-scale,  facility- wide  implementation  becomes  feasible  and  more  likely,  hospital 
administrators  have  become  interested  in  quantifying  the  costs  and  benefits  of  this  new 
technology.  From  a  technical  standpoint,  PACS  clearly  can  replace  film-based  technology  (Huda, 
Honeyman,  Frost  &  Staab,  1996).  However,  it  is  important  to  consider  the  economic  impact  of 
implementing  PACS.  Past  experience  shows  that  the  benefits  of  new  technology  may  be 
overshadowed  by  substantial  increases  in  cost  (Evans,  1989).  Indeed,  implementation  of  PACS 
has  sometimes  been  postponed  due  to  the  lack  of  a  clear  demonstration  of  the  cost-effectiveness 
of  this  technology  (Straub  &  Gur,  1990).  Radiology’s  reliance  on  costly  technology  makes  it  an 
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obvious  target  for  budget  conscious  health  care  administrators  (Becker  &  Arenson,  1994).  There 

is  increased  interest,  therefore,  in  conducting  economic  evaluations  of  PACS  to  determine  their 

cost-effectiveness.  Hilsenrath,  Smith  and  Berbaum  (1994)  state: 

Profitability  is  increasingly  used  as  a  criterion  for  investment  in  health 
care,  owing  to  economic  pressures  that  limit  the  scope  for  cross¬ 
subsidization.  Profits  will  erode  if  capital  expenditures  do  not 
ultimately  reduce  costs,  or  alternatively,  lead  to  an  equivalent  or 
greater  increase  in  revenues. 

Traditional  cost  analyses  of  PACS  have  focused  solely  on  comparing  the  capital  and 
operating  costs  of  implementing  this  system  with  the  direct  (actual)  costs  of  a  typical  film-based 
system  (Straub  &  Gur,  1990).  The  major  savings  expected  from  a  PACS  may  be  attributed  to 
supply  (film  and  chemicals),  film-handling  personnel,  and  storage  space  (Langlotz  et  al.,  1995; 
Huda  et  al.,  1996).  Additional  savings  may  be  realized  from  reduction  in  the  amount  of 
hazardous  material  produced  (e.g.,  spent  developer  and  fixer  solutions)  and  its  disposal.  Most 
studies,  however,  have  failed  to  consider  hidden  or  indirect  costs  of  film-based  systems  that  may 
result  from  inefficiencies  such  as  lost  films,  duplicated  (repeated)  examinations,  and  lack  of 
timely  access  to  diagnostic  images  and  reports  (Straub  &  Gur,  1990,  Langlotz  et  al.,  1995). 
Indirect  costs  are  difficult  to  quantify  (Strickland,  1996),  but  may  be  assessed  by  surveying  the 
medical  staff.  Straub  and  Gur  (1990,  p.614)  state  that  “input  from  physicians  can  provide 
valuable,  although  not  empirical,  evidence  of  the  general  importance  and  magnitude  of  the 
indirect  costs  of  the  system.”  Indeed,  significant  cost  savings  may  be  realized  by  implementation 
of  PACS,  as  a  result  of  more  timely  access  to  images  and  reports,  permitting  more  timely 
decisions  regarding  patient  treatment.  The  resultant  increase  in  efficiency  could  substantially 
decrease  costs,  an  attractive  outcome  in  a  managed  care  environment.  Studies  by  Tucker  et  al. 
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(1995a),  Langlotz  et  al.  (1995),  and  Huda  et  al.  (1996)  determined  that  the  cost  of  a  PACS  is, 
indeed,  offset  by  savings  from  eliminating  the  cost  of  film  production  and  management. 

Langlotz  et  al.  (1995)  described  three  types  of  economic  analyses:  (a)  cost-benefit,  (b) 
cost-effectiveness,  and  (c)  cost  identification.  Cost-benefit  analysis  compares  the  costs  of 
medical  intervention  to  its  benefits,  in  monetary  units  (e.g.,  dollars).  Cost-effectiveness  analysis 
is  similar,  but  costs  and  benefits  are  not  necessarily  expressed  in  the  same  units,  since  medical 
outcomes  often  cannot  be  expressed  in  units  of  currency.  Results  of  cost-effectiveness  analysis 
are  most  often  reported  as  a  ratio  of  the  number  of  dollars  spent  per  unit  of  outcome  (e.g.,  dollars 
per  year  of  life  saved).  Cost-identification  analysis  is  less  complex;  it  simply  lists  costs  without 
consideration  of  outcomes.  Cost-identification  is  useful  in  comparing  the  costs  of  alternative 
methods  for  providing  the  same  service.  Most  economic  studies  of  PACS  are,  therefore, 
characterized  as  cost-identification  analyses  that  examine  direct  costs  in  a  radiology  department 
(Langlotz  et  al.,  1995).  Department  expenditures  are  used  to  estimate  the  cost  of  film-based 
radiography,  and  an  analysis  of  departmental  operations  affords  an  estimate  of  equipment, 
supplies,  and  personnel  costs  for  the  operation  of  a  PACS.  The  direct  costs  of  PACS  and  film- 
based  radiology  are  presented  in  Table  2  and  Table  3. 

Purpose 

The  purpose  of  the  current  study  is  (1)  to  identify  and  measure  the  direct  costs  of 
complete  implementation  of  the  PACS  at  WRAMC  using  cost  identification  analysis;  (2)  to 
compare  the  direct  costs  of  the  PACS  and  conventional  film-based  radiology;  (3)  to  determine 
the  level  of  healthcare  provider  satisfaction  with  the  PACS;  and  (4)  to  present  the  findings  and 
recommendations  to  the  Chief,  Department  of  Radiology,  to  assist  him  in  determining  the  cost- 
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efficiency  of  the  PACS  at  WRAMC  and  to  suggest  future  courses  of  action  regarding  expansion 
of  the  sytem  at  WRAMC. 

The  working  hypothesis  of  this  study  is  that  the  PACS  at  WRAMC  is  a  cost-efficient 
alternative  to  film-based  radiology  (based  on  cost  identification  analysis)  and  that  healthcare 
providers  at  WRAMC  are  satisfied  with  the  system.  Further,  overall  satisfaction  with  the  PACS 
is  a  function  of  ten  specific  factors  (the  first  ten  questions  on  the  survey  instrument). 

Method  and  Procedures 

This  research  was  a  retrospective,  predictive  analysis  of  the  PACS  system  at  WRAMC. 

The  PACS  at  WRAMC  was  assessed  in  a  manner  patterned  after  that  described  by  Straub 
and  Gur  (1990).  Two  methods  were  used  in  the  assessment  of  the  PACS  at  WRAMC:  (1)  cost 
identification,  and  (2)  healthcare  provider  satisfaction.  Direct  costs  of  PACS  and  traditional  film- 
based  radiology,  as  described  in  Table  2  and  Table  3,  were  determined  from  a  quotation  from 
Agfa  Medical,  Division  of  Bayer,  Inc.  and  historical  records  (purchase  requests  and  document 
registers)  supplied  by  the  Department  of  Radiology.  Provider  satisfaction  with  the  PACS  was 
measured  using  the  survey  instrument  in  Appendix  A. 

A  survey  must  include  a  representative  sample  of  the  population  studied.  In  the  present 
study,  the  population  consisted  of  all  healthcare  providers  who  may  utilize  the  PACS  at 
WRAMC.  The  sample  included  providers  assigned  to  departments  that  are  major  users  of  the 
PACS. 

The  most  appropriate  applications  of  surveys  are  those  in  which  the  sample  is  composed 
of  respondents  who  are  "uniquely  qualified  to  provide  the  desired  information"  (Cooper  & 
Emory,  1995,  p.  270).  Therefore,  the  instrument,  an  eleven-item  questionnaire,  was  distributed  to 
practitioners  assigned  to  departments  that  are  major  users  of  the  system:  Radiology,  Emergency, 
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Medicine,  Surgical  Intensive  Care,  Cardiothoracic  Surgery,  Pulmonary,  and  Urology.  The  survey 
questions  were  closed  end.  Responses  were  based  on  a  modified  five-point  Likert  scale  (Likert, 
1932),  substituting  numeric  values  for  semantic  labels.  Numeric  values  ranged  from  1  ("strongly 
agree")  to  5  ("strongly  disagree").  Respondents  with  no  opinion  on  a  particular  question  were 
afforded  the  opportunity  to  select  a  neutral  midpoint  (a  score  of  3). 

In  the  development  and  administration  of  a  survey  tool,  the  issues  of  validity  and 
reliability  must  be  addressed.  A  method  to  determine  content  validity  is  to  have  a  panel  of 
individuals  make  a  judgement  regarding  the  extent  to  which  an  instrument  provides  adequate 
coverage  of  the  topic  under  study  (Cooper  &  Emory,  1995).  Therefore,  the  instrument  was 
evaluated  and  critiqued  by  representatives  from  the  Department  of  Radiology  and  reviewed  by  a 
representative  from  the  Department  of  Nursing  Education  and  Research. 

Content  Validity  of  the  survey  was  evaluated  using  procedures  described  by  Waltz, 
Strickland  and  Lentz  (1984).  When  two  judges  are  employed,  the  Index  of  Content  Validity 
(CVI)  is  used  to  quantify  the  degree  of  agreement  between  the  experts.  To  compute  the  CVI,  the 
experts  are  given  the  objectives  of  the  survey  and  the  survey  items.  They  are  requested  to  rate  the 
relevance  of  each  item  to  the  objectives  using  a  four  point  scale:  (1)  not  relevant,  (2)  somewhat 
relevant,  (3)  quite  relevant,  and  (4)  very  relevant.  The  CVI  is  defined  as  the  proportion  of  items 
given  a  rating  of  quite  relevant  and  very  relevant  by  both  of  the  raters.  If  all  items  receive  quite 
relevant  or  very  relevant  ratings  by  both  raters,  the  CVI  is  1 .00,  indicating  perfect  interrater 
agreement.  A  CVI  of  .70  was  required  to  establish  content  validity  of  the  survey  instrument.  A 
copy  of  the  instrument  used  to  establish  content  validity  is  presented  in  Appendix  B.  Survey 
questions  were  grouped  into  three  categories:  benefits  to  the  diagnostician  (radiologist),  benefits 
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to  the  referring  physician,  and  benefits  to  the  patient.  Judges  were  requested  to  evaluate  the 
relevance  of  each  question  to  the  appropriate  category. 

A  measure  that  affords  consistent  results  is  considered  reliable.  A  reliable  measure  is  free 
from  random  or  unstable  error  (Cooper  &  Emory,  1995).  Internal  reliability  can  be  assessed 
using  Cronbach's  coefficient  alpha.  The  acceptable  criterion  for  high  internal  reliability  is 
Cronbach  a  >  .70  (Cronbach,  1951).  This  test  was  performed  to  determine  the  reliability  of  the 
survey  instrument. 

Pretesting  permits  identification  of  problems  or  weaknesses  in  a  survey  instrument  before 
data  collection  begins.  Colleagues,  respondent  surrogates,  or  actual  respondents  evaluate  and 
refine  the  questionnaire.  Pretesting  of  the  survey  instrument  was  accomplished  by  surveying 
radiologists  at  WRAMC  and  soliciting  feedback. 

Data  from  the  survey  was  entered  into  SPSS®,  a  statistical  software  package.  Missing 
values  were  replaced  with  the  series  mean.  Descriptive  statistics  (means  and  standard  deviations) 
were  computed  for  all  continuous  variables  (all  eleven  questions  on  the  survey  instrument).  A 
correlation  matrix  was  constructed;  correlations  between  the  mean  scores  of  providers’  responses 
to  the  first  ten  survey  questions  and  the  single  question  of  overall  satisfaction  (question  11  on  the 
survey  instrument)  were  determined.  A  stepwise  multiple  regression  analysis  was  accomplished 
to  determine  the  contribution  of  each  of  the  ten  items  to  overall  satisfaction.  The  dependent 
variable  was  overall  satisfaction  and  the  independent  variables  were  each  of  the  other  ten  items 
included  in  the  survey  instrument.  An  alpha  level  of  .05  was  selected  for  the  regression  analysis. 
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Results 

Equipment  required  to  complete  implementation  of  the  PACS  at  WRAMC  is  presented  in 
Table  4,  which  was  derived  from  a  quotation  to  the  Department  of  Radiology,  WRAMC,  from 
Agfa  Medical,  Division  of  Bayer,  Inc.,  dated  31  March  1998.  The  annual  cost  to  complete 
implementation  of  the  PACS  at  WRAMC  is  presented  in  Tables  5  and  6.  These  data  were  also 
supplied  by  Agfa  Medical,  Divison  of  Bayer,  Inc.  The  annual  cost  of  continuing  to  use  film- 
based  radiology  is  presented  in  Table  7,  which  is  based  on  data  supplied  by  the  Department  of 
Radiology,  WRAMC.  The  total  annual  operating  cost  of  the  PACS  for  the  first  year  is 
$2,485,924.80.  In  subsequent  years,  the  total  annual  operating  cost  declines  from  a  high  of 
$2,774,724.80  in  year  2  to  a  low  of  $1,812,861 .80  in  year  8  (due  to  changes  in  the  cost  of 
maintenance  and  a  full  time  on-site  field  service  engineer).  In  contrast,  the  total  annual  operating 
cost  of  film-based  radiology  amounts  to  $913,665.97  (based  on  fiscal  year  1997  data). 

The  Index  of  Content  Validity  (CVI)  for  the  survey  instrument  was  .95,  indicating  a  high 
degree  of  interrater  agreement.  This  value  exceeded  the  minimum  value  of  .70  required  to 
establish  content  validity. 

The  survey  was  administered  during  the  period  9  February  -  27  March  1998  to  healthcare 
providers  assigned  to  the  departments  previously  described.  Distribution  of  the  survey  to 
practitioners  working  in  departments  that  are  major  users  of  the  PACS  assured  that  the  sample 
was  representative  of  the  population.  Of  132  surveys  distributed,  119  were  completed, 
representing  a  90%  rate  of  return.  The  high  rate  of  return  may  be  attributed  to  the  fact  that  the 
surveys  were  distributed,  completed,  and  returned  during  scheduled  departmental  staff  meetings. 
The  high  rate  of  return,  the  low  rate  of  missing  data  (.45%)  and  the  representativeness  of  the 
sample  contribute  to  the  reliability  and  usefulness  of  the  survey  instrument. 
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Cronbach's  coefficient  alpha  for  the  survey  instrument  was  determined  to  be  .7928.  This 
exceeded  the  acceptable  criterion  for  high  internal  reliability. 

Demographics  of  the  survey  participants  are  presented  in  Table  8.  The  majority  are  male 
(83%)  and  are  on  active  duty  in  the  military  (92%),  with  a  varying  degree  of  experience  as 
medical  care  providers. 

Descriptive  statistics  for  each  of  the  eleven  continuous  variables  are  presented  in  Table  9. 
Generally,  survey  participants  expressed  moderate  to  strong  agreement  with  the  survey  items, 
although  degree  of  agreement  ranged  the  entire  spectrum  from  “strongly  agree”  to  “strongly 
disagree”  for  some  items.  They  agreed  highly  that  the  PACS  provides  rapid  access  to 
radiographic  images  (question  1,  M  =  1.47).  Likewise,  they  agreed  that  the  ability  to  review 
multiple  images  simultaneously  in  more  than  one  location  (that  is,  the  presence  of  multiple 
viewing  stations)  is  important  in  their  clinical  practice  (question  6,  M  =1.76).  The  participants 
are  confident  in  their  diagnoses  using  digital  images  (question  10,  M  =  1.98)  and  they  agree  that 
the  PACS  improves  the  quality  of  care  delivered  (question  4,  M  =  2.02).  Further,  the  providers 
agree  that  the  PACS  increases  the  efficiency  of  clinical  practice  (question  5,  M  =  2.03).  They 
expressed  moderate  agreement  with  the  remainder  of  the  survey  items.  Overall  satisfaction  with 
the  performance  of  the  PACS  at  WRAMC  was  moderate  (question  1 1,  M  =  2.17). 

Overall  satisfaction  by  department  varied  widely  and  is  presented  in  Table  10.  Some 
departments,  including  Radiology,  Emergency,  Surgical  Intensive  Care,  and  Medicine  are  quite 
satisfied  with  the  PACS.  Other  departments  are  less  satisfied. 

Overall  satisfaction  also  varied  by  type  of  healthcare  provider  (see  Table  1 1).  For 
purposes  of  this  analysis,  survey  participants  were  categorized  into  two  groups.  One  group, 
labeled  "student,"  included  those  enrolled  in  undergraduate  or  graduate  medical  education  (i.e.. 


Cost  Identification  and  Benefits 


27 


medical  students,  interns,  and  residents).  The  other  group,  labeled  "staff,"  included  attending  and 
staff  physicians,  as  well  as  other  healthcare  providers  (e.g.,  physician  assistants).  Survey 
participants  in  the  "student"  category  exhibited  a  higher  level  of  overall  satisfaction  (M  =  2.03) 
than  did  those  in  the  "staff'  category  (M  =  2.44). 

Correlations  between  mean  scores  of  provider  responses  to  questions  1  through  10  and 
overall  satisfaction  (question  1 1)  are  presented  in  Table  12.  The  correlation  matrix  reveals  a 
positive  relationship  between  the  mean  scores  of  providers’  responses  to  all  but  one  of  the  first 
ten  survey  questions  and  the  single  question  of  overall  satisfaction.  (There  is  a  weak  negative 
correlation  with  question  2.)  All  correlations  with  question  1 1  are  significant  at  the  .01  level 
except  for  question  2.  There  is  a  strong,  positive  relationship  between  overall  satisfaction  and 
question  1  (r_=  .399),  question  4  (r_=  .433),  question  5  (r  =  .513),  question  8  (r_=.585),  question  9 
(r_=  .505),  and  question  10  (r  =  .537). 

Results  of  the  stepwise  multiple  regression  analysis  are  presented  in  Tables  13, 14  and 
15.  Analysis  of  the  regression  revealed  that  the  most  significant  factor  associated  with  overall 
satisfaction  with  the  PACS  at  WRAMC  was  the  "user  friendliness"  (i.e.,  the  ease  of  use)  of  the 
system  (question  8),  and  this  resulted  in  a  shared  variance,  R2  =  .342  (a  highly  predictive  value) 
with  F(l,117)  =  60.812,  p<  .0001.  Three  other  variables  that  contributed  significantly  to  the 
shared  variance  were  questions  10,  5  and  9,  which,  when  included  in  model  4,  resulted  in  a 
shared  variance,  R2  =  .557  with  F  (4, 1 14)  =  35.847,  p  <  .0001 .  Thus,  overall  satisfaction  with  the 
PACS  at  WRAMC  can  be  predicted  from  a  model  consisting  of  provider  perceptions  of  (1)  the 
"user  friendliness"  of  the  PACS,  (2)  its  effect  on  the  efficiency  of  clinical  practice,  (3)  its 
reliability,  and  (4)  providers'  confidence  in  diagnosis  using  the  PACS. 
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Responses  to  two  additional  survey  questions  are  presented  in  Table  16.  The  majority  of 
providers  surveyed  (87%)  desire  to  have  a  dedicated  review  station  located  in  their  clinical  areas 
and  92%  would  like  to  have  radiographic  images  and  reports  displayed  on  their  personal 
computers. 

Provider  written  comments  are  presented  in  Table  17.  Most  providers  expressed  a  desire 
to  have  more  viewing  stations  located  in  close  proximity  to  their  clinical  areas.  Two  providers 
reported  difficulty  in  retrieving  archived  images.  One  expressed  a  keen  desire  to  have  the 
opportunity  to  view  images  and  reports  on  his  personal  computer. 

Discussion 

Although  the  cost  identification  methodology  employed  in  this  study  is  not  sophisticated, 
the  data  do  identify  the  key  financial  issues  associated  with  further  implementation  of  the  PACS 
at  WRAMC.  Existing  capital  equipment  represents  a  "sunk"  cost  and  is  not  included  in  this 
analysis. 

Huda  et  al.  (1996)  found  that  the  major  savings  that  can  be  realized  by  implementing  a 
PACS  are  from  supplies  (film  and  chemicals)  and  employee  savings.  The  present  study  reveals 
that  film-based  radiology  at  WRAMC  costs  $548,151.97  annually  in  supply  and  production  costs 
and  $172,314.00  in  staff  costs  (see  Table  7).  No  new  capital  equipment  for  film-based  radiology 
has  been  proposed.  The  total  annual  supply,  production  and  personnel  cost  of  film-based 
radiology  at  WRAMC  is,  therefore,  $720,465.97. 

Complete  implementation  of  the  PACS  at  WRAMC  would  result  in  a  saving  of  4,830 
square  feet  of  storage  space  (film-based  radiology  and  PACS  require  4,878  and  48  square  feet, 
respectively).  The  annual  cost  of  one  square  foot  of  storage  space  ranges  from  $10  to  $140 
(Becker  et  al.,  1994).  In  the  metropolitan  Washington,  DC  area,  the  rental  price  for  commercial 
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property  varies  greatly,  but  has  been  reported  as  $40  per  square  foot  per  annum  in  downtown 
office  buildings  (Hunt  &  Weichert  Commercial  Realtors,  1998).  Therefore,  the  annual  storage 
cost  of  film-based  radiology  could  potentially  amount  to  $193,200.  Thus,  the  total  annual 
operating  cost  of  film-based  radiology  at  WRAMC  is  $913,665.97  (see  table  7). 

Cost-analyses  differ  in  reports  of  the  direct  costs  of  P  ACS  and  film-based  radiology,  and 
these  differences  cannot  be  explained  by  variations  in  the  size  of  the  medical  facility  studied  or 
the  number  of  examinations  performed  (Becker  &  Arenson  1994).  Variations  in  the  cost  of 
space,  material,  and  personnel  at  least  partially  account  for  the  differences.  Also,  the  purchase 
price  of  capital  equipment,  maintenance  costs,  and  amortization  costs  may  vary  in  different 
locations.  The  most  substantial  cost  of  the  PACS  in  the  present  study  is  that  for  maintenance  and 
a  full  time  service  engineer. 

Studies  by  Tucker  et  al.  (1995a)  and  Huda  et  al.  (1996)  found  that  savings  from  the 
elimination  of  conventional  film  production  and  management  offset  the  total  cost  of  a  PACS.  In 
contrast,  the  present  study  determined  that,  although  the  PACS  does  afford  substantial  savings  in 
film  and  production  costs,  the  overall  total  annual  cost  of  complete  implementation  of  the  PACS 
at  WRAMC  exceeds  the  total  annual  cost  of  film-based  radiology  by  $1,572,258.83  in  year  1  and 
between  $1,861,058.83  and  $899,195.83  in  years  2  through  8.  Huda  et  al.  (1996)  developed 
workstations  in-house  at  reduced  cost  ($50,000  each)  and  did  not  include  the  cost  of  purchasing 
remote  review  stations,  since  they  planned  to  continue  use  of  existing  image  display  stations  that 
were  installed  previously.  (These  represented  "sunk"  costs.)  WRAMC  will  require  the  purchase 
of  nine  D2  Diagnostic  workstations  (equivalent  to  existing  2C  viewing  stations)  for  the 
Pulmonary  and  Orthopaedic  Departments,  twelve  D4  workstations  (equivalent  to  existing  4C 
workstations)  for  Radiology,  forty-two  R2  clinical  review  stations,  and  nine  R1  technologist 
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review  stations  at  an  annual  cost  of  $844,505.88  (see  Tables  5  and  6).  This  would  enable 
providers  to  view  images  in  their  respective  clinics,  a  highly  desirable  procedure  as  indicated  by 
results  of  the  survey. 

Survey  results  indicate  that,  although  the  diagnostic  accuracy  of  digital  images  may  not 
equal  or  exceed  that  of  film-based  techniques  (question  2),  physicians  at  WRAMC  are, 
nonetheless,  confident  in  their  diagnoses  using  digital  techniques.  The  results  indicate  a  moderate 
overall  satisfaction  with  the  PACS  at  WRAMC.  Physicians  believe  that  the  system  provides 
rapid  access  to  radiographic  images  and  enhances  the  quality  of  care  delivered.  They  greatly 
value  the  ability  to  review  images  simultaneously  (i.e.,  the  presence  of  multiple  viewing 
stations).  Most  desire  the  convenience  of  having  a  dedicated  review  station  in  their  clinic  and 
would  like  to  be  able  to  view  reports  and  images  on  their  personal  computers. 

As  noted  above,  overall  satisfaction  by  department  varied  greatly.  The  departments  of 
Radiology,  Emergency,  Surgical  Intensive  Care  and  Medicine  enjoy  ready  access  to  viewing 
stations  in  their  clinics  and  hence  exhibit  a  high  degree  of  satisfaction  with  the  PACS.  Access 
for  other  departments,  especially  and  Urology,  is  problematic;  they  do  not  have  easy  access  to 
viewing  stations  and  so  a  lower  level  of  satisfaction  may  be  expected. 

Overall  satisfaction  also  varied  by  type  of  provider.  Students,  who  are  usually  younger 
and  are  part  of  a  generation  that  grew  up  with  computers,  can  be  expected  to  be  more  receptive  to 
the  introduction  of  new  technology,  including  PACS.  Seasoned  staff  physicians,  in  contrast,  may 
face  a  steep  "learning  curve."  These  findings  are  in  agreement  with  those  of  Smith  et  al.  (1995), 
who  reported  that  radiology  residents  at  MAMC  were  nearly  unanimous  in  their  approval  of 
PACS.  Staff  radiologists  at  that  facility  who  use  the  workstations  most  frequently  are  typically 
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older  and  had  never  used  a  computer  before  the  PACS  was  installed.  Nonetheless,  even  senior 
clinicians  have  learned  to  use  the  system  effectively. 

The  correlation  results  indicate  that  provider  satisfaction  with  the  PACS  is  strongly 
correlated  with  provider  perceptions  regarding  ready  access  to  radiographs,  the  quality  of  care 
delivered,  clinical  efficiency,  ease  of  use,  reliability,  and  confidence  in  arriving  at  an  appropriate 
diagnosis  using  digital  images. 

The  current  study  determined  that,  based  on  regression  analysis,  the  major  factors 
contributing  to  overall  satisfaction  with  the  PACS  at  WRAMC  are  its  ease  of  use,  its  effect  on 
the  efficiency  of  clinical  practice,  its  reliability,  and  providers'  confidence  in  diagnosis  using  the 
PACS. 

Leckie  et  al.  (1993)  reported  that  a  consistent  observation  of  all  new  users  of  a  PACS  is 
that  the  system  is  easy  to  understand  and  learn.  Tucker  et  al.  (1995a)  also  found  that  the  response 
from  providers  using  viewing  stations  was  uniformly  positive.  They  liked  the  convenience  and 
easy  access  to  images  and  reports  provided  by  the  PACS.  Healthcare  providers  at  WRAMC  agree 
with  this  observation.  Smith  et  al.  (1995)  state  that  a  training  program  employing  vendor- 
supplied  computer  technicians  and  trainers  in  a  small  group  setting  affords  clinicians  a  basic 
understanding  of  PACS  and  workstation  functions.  He  concludes  that  additional  "hands-on" 
practice,  coaching  from  knowledgeable  colleagues,  and  time  and  experience  will  eventually  lead 
to  full  utilization  of  the  system  in  clinical  practice. 

Tucker  et  al.  (1995a)  reported  that  one  of  the  direct  benefits  of  a  PACS  is  improved 
efficiency  for  physicians,  technologists  and  staff.  They  stated  that  repeated  examinations  may 
result  from  improper  technique,  positioning  errors,  lost  or  misfiled  radiographs,  or  poor  technical 
film  quality.  The  rate  of  missing  radiographic  films  may  be  as  high  as  30%  in  some  hospitals 
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(Osteaux,  Van  den  Broeck,  Verhelle  &  de  Mey,  1996).  This  may  have  profound  medical  and 
medico-legal  implications.  Repeated  examinations,  regardless  of  cause,  are  not  desirable  (Straub 
&  Gur,  1990).  Repeated  examinations  expose  patients  to  unnecessary  ionizing  radiation  and 
treatment  delay,  and  can  result  in  lost  revenue  and  inefficiency  for  the  hospital.  Digital 
radiography  can  reduce  the  number  of  repeated  examinations  by  eliminating  over-  and 
underexposure  problems,  and,  by  virtue  of  its  image  management  capabilities,  prevent 
mishandling  and  film  loss.  Therefore,  with  the  exception  of  repeated  studies  due  to  positioning 
errors,  PACS  can  effectively  eliminate  common  radiographic  problems  and  improve  the 
efficiency  of  the  radiology  department.  Tucker  et  al.  (1995a)  reported  that  at  the  University  of 
Alabama  Hospitals  and  Clinics,  the  frequency  of  repeated  examinations  decreased  from  15%  to 
3%  upon  implementation  of  computed  radiography.  Smith  et  al.  (1995)  estimate  a  time  savings 
of  one  hour  per  week  per  clinician  due  to  enhanced  image  and  report  availability  with  PACS. 
This  translates  to  saving  20,000  hours  annually  in  a  hospital  that  employs  400  physicians! 
Physicians  at  WRAMC  agree  that  the  PACS  increases  the  efficiency  of  their  clinical  practice  and 
improves  the  quality  of  care  delivered. 

The  third  major  factor  contributing  to  overall  satisfaction  with  the  PACS  at  WRAMC  is 
system  reliability.  A  hospital  operates  24  hours  per  day  and  past  experience  indicates  that  even  a 
95%  system  availability  is  barely  sufficient  for  user  acceptance  (Smith  et  al.,  1995).  Interruption 
of  clinical  services  could  have  dire  life  or  death  consequences.  Smith  et  al.  (1995)  recommend 
that  a  PACS  contract  should  require  99%  system  availability  and  include  detailed  plans  for  crisis 
management  and  operations  when  the  system  is  "down."  They  report  a  high  degree  of  reliability 
(99.7%)  of  the  PACS  at  MAMC.  Providers  at  WRAMC  moderately  agree  that  the  PACS  is 
reliable  (M  =  2.21). 
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The  other  major  factor  contributing  to  overall  satisfaction  is  providers'  confidence  in 
diagnosis  using  the  PACS.  Healthcare  providers  at  WRAMC  are  dedicated  to  providing  optimal 
care  for  all  beneficiaries,  and  arriving  at  an  appropriate  diagnosis  is  key  to  providing  top  quality 
care.  Providers  at  WRAMC  are  confident  in  their  diagnoses  using  the  PACS  and  they  believe 
that  the  system  enhances  the  quality  of  care  provided  at  this  institution. 

Conclusion  and  Recommendations 

PACS  can  be  considered  an  extension  of  a  hospital’s  information  system  (HIS),  in  which 
data  may  be  obtained,  stored,  retrieved  and  displayed  in  digital  format.  However,  the  volume  of 
data  created  by  a  PACS  is  typically  much  greater  than  that  of  a  typical  HIS.  Additionally,  there 
are  requirements  for  high  quality  viewing  stations  and  transmission  speed  that  is  several  orders 
higher  in  magnitude  than  that  required  for  conventional  HIS.  As  a  result,  a  PACS  requires  more 
high-technology  equipment  and,  therefore,  a  substantial  financial  investment  (Van  Gennip  & 
Bakker,  1993).  Hence,  although  prototypes  were  available  in  the  early  1980’s,  routine  clinical 
use  of  PACS  has  developed  slowly.  It  is  no  surprise  that  hospital  administrators  must  weigh  the 
costs  and  benefits  of  PACS  before  deciding  to  invest  in  this  technology. 

Straub  and  Gur  (1990)  reported  two  major  barriers  to  implementation  of  a  PACS: 
cost/benefit  concerns  and  the  belief  that  the  diagnostic  accuracy  of  PACS  may  not  be  comparable 
to  that  attained  using  conventional  film.  The  cost  identification  analysis  in  the  present  study 
focused  on  identifying  direct  costs  associated  with  the  PACS  and  conventional  film-based 
radiology.  The  study  confirms  that  complete  implementation  of  the  PACS,  as  proposed,  is  indeed 
expensive.  However,  Straub  and  Gur  (1990)  clearly  demonstrated  that  significant  future  savings 
may  lie  in  avoidance  of  indirect  (hidden)  costs  resulting  from  rapid,  timely  access  to  diagnostic 
images  and  reports,  and  enhanced  efficiency  of  physicians'  clinical  practice.  The  present  study 
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confirmed  these  findings:  physicians  at  WRAMC  agree  that  the  PACS  provides  rapid  access  to 
radiographic  images  and  improves  efficiency.  Further,  they  believe  that  the  PACS  improves  the 
quality  of  care  rendered  to  patients.  Unfortunately,  indirect  costs  are,  at  best,  difficult  to  quantify. 
Hence,  opinions  regarding  the  total  costs  and  savings  of  PACS  vary  widely  (Van  Gennip  & 
Bakker,  1993).  However,  a  PACS  does  offer  the  promise  of  efficient  management  of  a  hospital’s 
increasingly  large  and  complex  information  base  in  radiology.  Future  studies  should  attempt  to 
quantify  the  indirect  costs  that  result  from  less  than  ideal  film  archival  and  communication 
systems.  "Only  then  can  the  potential  impact  of  digital  PACS  on  total  cost  be  appreciated,  and 
only  then  might  the  financial  justification  for  PACS  implementation  be  rationalized"  (Straub  & 
Gur,  1990,  p.  616).  Because  of  high  cost  estimates,  many  hospitals  are  reluctant  to  implement  a 
PACS.  Van  Gennip  and  Bakker  (1993)  suggest  that  in  addition  to  identifying  costs  and  benefits 
of  a  PACS,  and  before  investing  in  a  PACS,  healthcare  institutions  should  seriously  consider 
whether  a  it  is  the  best  method  to  solve  problems  with  storage  and  distribution  of  medical  image 
information. 

Regarding  diagnostic  accuracy  of  PACS,  findings  in  the  literature  are  mixed.  According 
to  Huda  et  al.  (1996),  limitations  of  computed  radiography  include  inferior  spatial  resolution  and 
higher  image  noise  when  compared  to  film-based  radiography  using  the  same  radiation  dose. 
Likewise,  Stormer,  Boole,  Sund,  Weller,  and  Gitlin  (1997)  reported  that  conventional  films  were 
generally  perceived  to  be  of  better  quality  than  screen  images  and  that  radiologists  were  more 
confident  in  their  diagnoses  using  conventional  film.  In  contrast,  Kido,  Ikezoe,  Takeuchi,  and 
Kondoh  (1994)  found  no  statistically  significant  difference  in  performance  between  film  and 
digital  radiography.  Kimme-Smith,  Aberle,  Sayre,  Hart,  Greaves,  Brown,  Young,  Deseran, 
Johnson  and  Johnson  (1995)  found  no  statistically  significant  difference  except  when  computed 
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radiographs  were  underexposed.  Jonsson,  Hannesson,  Herrlin,  Jonsson,  Andersen,  and  Pettersson 
(1995)  reported  that  as  a  result  of  incorporation  of  smaller  phosphorous  particles  on  imaging 
plates  and  the  use  of  high-resolution  monitors,  digital  systems  provide  adequate  diagnostic 
accuracy.  The  present  study  confirmed  the  findings  of  the  latter  researchers:  physicians  at 
WRAMC  are  confident  in  their  diagnoses  using  digital  images. 

Because  of  constrained  resources,  especially  in  a  managed  care  environment,  with 
emphasis  on  minimizing  costs,  a  healthcare  institution  cannot  adopt  all  medical  innovations. 
Clinicians  and  healthcare  administrators  must,  therefore,  carefully  consider  costs  and  benefits  in 
the  selection  of  new  technology.  They  must  consider  effectiveness,  economy,  appropriateness, 
and  need.  In  sum,  adoption  of  a  given  technology  should  be  based  on  its  appropriateness  in 
solving  a  defined  problem. 

The  Chief,  Department  of  Radiology,  should  consider  the  following  when  contemplating 
complete  implementation  of  the  PACS  at  WRAMC: 

System  reliability  is  very  important  to  physicians  at  WRAMC  and  two  critical  issues 
must  be  considered.  First,  since  a  PACS  consists  of  a  multitude  of  components,  the  probability  of 
a  failing  component  is  high.  Second,  because  a  PACS  manages  important  patient  information, 
the  system  cannot  be  "down"  for  long  periods  of  time.  Redundancy,  therefore,  is  a  critical  factor 
in  meeting  reliability  requirements  and  can  greatly  enhance  system  performance  at  minimal  cost 
(Tucker  et  al.,  1995a).  In  addition,  spare  parts  should  be  readily  available. 

Viewing  stations  should  be  decentralized,  i.e.,  located  in  clinical  areas  throughout  the 
hospital,  to  provide  easy  access  to  the  system.  Distribution  of  viewing  stations  depends  upon  the 
level  of  clinic  activity,  the  time  required  to  review  images,  the  physical  layout  of  the  clinic,  and 
the  number  and  location  of  conference  rooms  (Smith  et  al.,  1992;  Leckie  et  al.,  1993).  Different 
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departments,  therefore,  will  have  varying  requirements  for  viewing  stations.  Smith  et  al.  (1995) 
report  that  at  MAMC,  the  approximate  ratio  is  3.4  physicians  per  viewing  station.  Viewing 
stations  should  be  located  in  areas  where  the  ambient  light  can  be  regulated  to  permit  optimum 
conditions  for  diagnostic  procedures  (Arenson,  Chakraborty  &  Seshadri,  1990).  The  survey  in 
the  present  study  indicated  that  the  presence  of  multiple  viewing  stations  is  a  major  concern  for 
healthcare  providers  at  WRAMC  and  has  a  significant  impact  on  satisfaction  with  the  PACS. 

A  “prefetching”  system  should  be  employed  to  send  images  from  the  ODJ  to  the  short¬ 
term  storage  unit  the  night  before  scheduled  examinations  or  consultations,  permitting  timely 
retrieval  the  next  day.  This  will  prevent  potential  “bottlenecks”  in  the  system  that  could  result 
during  periods  of  peak  demand,  if  images  were  instead  retrieved  on  an  ad  hoc  basis.  Providers  at 
MAMC  agree  that  a  “prefetching”  system  is  essential  for  successful  filmless  operations  at  that 
facility  (Smith  et  al.,  1995).  Likewise,  rapid  access  to  radiographic  images  is  very  important  to 
healthcare  providers  at  WRAMC. 

PACS  training  is  important  to  ensure  optimal  patient  care  and  to  satisfy  potential  ethical, 
risk  management,  and  medico-legal  requirements  (Protopapas,  Siegel,  Reiner,  Pomerantz,  Pickar, 
Wilson  &  Hooper,  1996).  Ideally,  training  should  accomplish  two  goals.  Firstly,  the  provider 
should  learn  how  to  use  the  system  to  expeditiously  retrieve  images  and  reports  and  to  display 
them  on  monitors.  Secondly,  the  provider  should  learn  to  use  the  tools  available  at  the  viewing 
station  to  optimize  the  images  for  interpretation,  in  order  to  derive  an  appropriate  diagnosis. 
PACS  training  should  be  accomplished  individually  or  in  small  groups,  so  that  it  may  be  tailored 
to  the  provider’s  individual  needs  and  specialty.  Computer-based  and  on-line  training  may  also 
be  considered.  A  continuous  PACS  training  program  is  especially  important  in  a  military  facility, 
due  to  the  high  rate  of  personnel  turnover  (Smith  et  al.,  1995). 
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One  workstation  should  be  dedicated  for  the  purpose  of  making  hardcopy  images,  using  a 
laser-film  printer  (see  Figure  5).  Due  to  the  high  turnover  of  active-duty  personnel  and  their 
families  to  other  military  installations,  most  of  which  do  not  have  a  PACS,  the  demand  for 
hardcopy  reprints  of  historical  images  may  be  substantial  (Smith  et  al.,  1995). 

A  mechanism  for  system  security  should  be  implemented  to  ensure  confidentiality  for 
patients.  This  will  help  to  prevent  unauthorized  access  to  patients’  demographic  and  examination 
information. 

An  open  architecture  network  design  permits  data  exchange  between  heterogeneous 
systems  (Hunag  &  Taira,  1992).  If  two  PACS  designs  cannot  communicate  with  each  other,  they 
become  isolated  and  cannot  be  combined  with  other  systems  to  develop  a  hospital-wide  or  health 
care  system-wide  PACS.  In  addition,  computer  and  communications  technology  is  changing 
rapidly  and  even  state-of-the-art  systems  soon  become  obsolete.  A  closed  architecture  network 
design  would  preclude  system  upgradability  and  inter-connectability  between  health  care 
facilities.  This  has  significant  implications  for  the  Walter  Reed  Health  Care  System  (WRHCS), 
which  consists  of  Walter  Reed  Army  Hospital,  Dewitt  Army  Community  Hospital,  Kimbrough 
Ambulatory  Care  Center  and  numerous  satellite  facilities.  Ideally,  the  PACS  should  provide  for 
seamless  transfer  of  digital  images  throughout  the  health  care  system,  thus  enhancing  referral  and 
consultation  procedures. 

Development  of  PACS  continues  to  be  one  of  the  most  challenging  tasks  in  the  field  of 
computer  engineering,  due  to  the  huge  volume  of  digital  image  data  produced,  necessitating  the 
introduction  of  novel  architectures  and  technology  (Meyer-Ebrecht,  1994).  Early  PACS  were 
extremely  expensive,  unreliable,  and  had  limited  image  storage  capability.  Today,  technology 
has  improved  markedly  and  PACS  can  improve  the  efficiency  of  delivering  healthcare,  moderate 
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operating  costs,  shorten  the  time  required  to  obtain,  distribute,  and  retrieve  radiographic  images, 
dramatically  reduce  the  number  of  repeat  examinations  and  lost  films,  and  produce  consistent 
image  quality  (Hynes  et  al.,  1997;  Tucker  &  McEachem,  1995b).  Sorting  through  a  film  jacket 
containing  25  films  requires  many  minutes.  Using  PACS,  all  25  images  may  be  displayed 
simultaneously  and,  with  the  click  of  a  mouse,  appropriate  ones  may  be  enlarged  for  comparison. 

A  PACS  can  reduce  or  eliminate  requirements  for  film  clerks,  dark-room  technicians, 
typists,  film  storage,  film  and  chemistry  costs  and,  theoretically,  lead  to  substantial  savings. 
However,  these  savings  may  be  offset  by  the  direct  cost  of  viewing  stations,  networks,  software, 
computers,  archives,  and  system  maintenance.  Nonetheless,  it  is  important  to  consider  the  harder- 
to-quantify  gains  that  may  be  realized  across  the  hospital  (Langlotz  et  al.,  1995)  due  to  shorter 
waiting  times,  more  timely  decision  making  (Dure-Smith  &  Fymat,  1997)  and  possibly  reduced 
length  of  stay  (Straub  &  Gur,  1990). 

The  trend  toward  PACS  is  inevitable  and  will  likely  be  complete  within  a  decade  (Hynes 
et  al.,  1997).  Some  experts  (Osteaux  et  al.,  1996)  believe  that  PACS  could  account  for  80%  of 
the  radiological  business  by  the  year  2000.  Delaying  implementation  is  probably  unwise,  since 
hospitals  that  offer  the  outreach  that  a  PACS  provides  will  have  a  competitive  edge  in  the 
marketplace. 

This  study  presented  the  major  benefits  that  can  be  anticipated  upon  complete 
implementation  of  a  PACS.  It  also  identified  the  direct  costs  of  the  system,  as  well  as  those 
associated  with  film-based  radiology.  A  PACS  can  provide  a  mechanism  to  solve  the  problem  of 
preserving,  transporting,  sorting,  reading,  storing  and  retrieving  radiographic  images  and  reports 
in  an  expeditious  and  efficient  manner.  In  addition,  through  implementation  of  a  PACS,  a 
hospital  can  expect  to  realize  significant  cost  savings  in  terms  of  supplies  (film  and  chemicals), 
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storage  space  and  personnel  required.  PACS  may  also  reduce  requirements  for  silver  recovery 
systems  and  disposal  of  hazardous  materials,  including  developer  and  fixer  solutions.  However, 
hardware,  software,  information  system,  and  maintenance  costs  of  a  PACS  are  high.  Therefore,  it 
is  the  task  of  clinicians  and  hospital  administrators  to  determine  if  a  PACS  is  the  best  method, 
consistent  with  their  budget,  to  solve  problems  with  storage  and  distribution  of  medical  image 
information,  resulting  in  a  net  incremental  improvement  in  the  health  of  the  population  served. 
They  must  assess  not  only  cost,  but  also  the  value  of  outcomes,  enhanced  productivity,  and  the 
societal  contribution  of  patients  that  have  been  restored  to  health  (Wells  et  al.,  1991). 


Cost  Identification  and  Benefits 


40 


Table  1 

Benefits  of  PACS 

Category 

Benefit 

Benefits  to  Diagnostician 

improved  access  to  current  patient  records 
improved  access  to  patient  history  records 
file  integrity  and  speed  of  retrieval 
better  diagnosis 

quicker  diagnosis/improved  productivity 

Benefits  to  Referring  Physician 

better  patient  management 
earlier  intervention 
better  outcome 
reduced  length  of  stay 
reduced  loss  of  films 

Benefits  to  Patient 

reduced  radiation  exposure 

shorter  examination  times 

fewer  re-takes 

reduced  inconvenience 

reduced  chance  of  adverse  reaction  from 

contrast  agents 

Benefits  to  Hospital 

a  j .r _ — : — . 

better  communication  with  physicians 
better  hospital  administration 
improved  morale  of  staff 

A  i «  A  nnoofimatit  rtf  D  A  ^  T 

Note.  Adapted  from  “Overview  of  Some  Methodological  Problems  in  Assessment  of  PACS,”  by 
B.L.  Crowe,  1992,  International  Journal  of  Biomedical  Computing,  30,  p.  185.  Copyright  1992 
by  Elsevier  Scientific  Publishers  Ireland  Ltd. 
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Table  2 
Cost  of  PACS 

1 .  System  (Capital)  Costs 

a.  Hardware 

(1)  Image  generating  apparatus  (CR  unit) 

(2)  Digital  interphases  (fiberoptic  transducers) 

(3)  Workstations  (including  monitors) 

(4)  Viewing  Stations 

(5)  Digitizer 

(6)  Cabling 

b.  Software 

2.  Maintenance  (engineering  support)  for  CR,  diagnostic  displays,  archive  and  server 

3.  Storage  Media  (optical  disks)  and  "Jukebox" 

4.  Archive  Loading 

5.  Supplies 

6.  Personnel  (including  PACS  technicians) 

7.  Space  for  archive  _ 

Note.  Adapted  from  "A  Cost-Analysis  of  Computed  Radiography  and  Picture  Archiving  and 
Communication  Systems  in  Portable  Radiology,"  by  W.  Huda,  J.  C.  Honeymoon,  M.  M.  Frost, 
and  E.V.  Staab,  1996,  Journal  of  Digital  Imaging,  9  (1)  p.  41. 
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Table  3 

Cost  of  Film-Based  Radiology 

1 .  System  (Capital)  Costs 

(a)  X-Ray  apparatus 

(b)  Developers  (automatic,  manual) 

(c)  Viewboxes 

2.  Film 

3.  Chemicals 

4.  Supplies 

5.  Miscellaneous  (e.g.  film  jackets) 

6.  Maintenance 

7.  Personnel  (including  film  librarian  and  darkroom  technician) 

8.  Space  for  archive _ 

Note.  Adapted  from  "A  Cost- Analysis  of  Computed  Radiography  and  Picture  Archiving  and 
Communication  Systems  in  Portable  Radiology,"  by  W.  Huda,  J.  C.  Honeymoon,  M.  M.  Frost, 
and  E.V.  Staab,  1996,  Journal  of  Digital  Imaging,  9  (1)  pp.  41-42. 


Cost  Identification  and  Benefits 


43 


Table  4 

Equipment  Required  to  Complete  Implementation  of  P ACS  at  WRAMC  (Includes  Hardware, 

Software,  Cabling,  and  Management) _ 

Item  Quantity 

QC  Workstation  Workflow  Manager  5 

Large  AS3000-DLT  Base  Package  1 

DLT  Jukebox  Media  10 

50  GB  External  RAID  for  AS3000  Packages  3 

Large  OS3000  Package  1 

IMP  AX  RS  3000-2P-2K  (D2)  Diagnostic  Workstation  (Pulmonary  &  Orthopaedics)  9 

IMP  AX  RS  3000-4P-2K  (D4)  Diagnostic  Workstation  (Radiology)  12 

IMPAX  RS  5 000-2- IK  (R2)  Clinical  Review  Station  42 

IMP  AX  RS  5000- 1  -1 K  (R1 )  Technologist's  Review  Station  9 

Web  1000  Server  &  30  Licenses  (each  server  supports  100  concurrent  users)  1 

Modem  with  Cable  1 

RIS  Gateway/Interface  1 

DIN-PACS  Embedded  Network  1 

DIN-PACS  Embedded  RIS  with  Training  1 

AFC  Industries  Work  Surfaces  with  Chair  for  all  Stations  1 

Site  Management  1 

Project  Management  1 

Field  Engineer  on-site  Acceptance  1 
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Table  5 

Annual  Cost  of  P ACS  at  WRAMC  (First  Year) 

Item  Total  Cost  ($) 


Equipment*  770,287.00 

Installation*  48,953.50 

Site  Preparation*  8,337.38 

Uninterruptable  Power  Supplies*  1 0,000.00 

CDRLs*  6,928.00 

TOTAL  ANNUAL  EQUIPMENT  COST  844,505.88 

Maintenance  1,634,699.00 

Full  Time  on-site  Field  Service  Engineer  (included  in  maintenance  cost) 

TOTAL  ANNUAL  EQUIPMENT,  MAINTENANCE, 

AND  PERSONNEL  COSTS  2,479,204.80 

Space  for  Archive  (square  feet) 

Optical  Disk  Jukebox  32 

ISU  12 

Personal  Computers  (2)  4 

Total  Square  Footage  48 

TOTAL  ANNUAL  STORAGE  COST  (@  $140  per  square  foot)  6,720.00 

TOTAL  ANNUAL  COST  OF  PACS  (First  Year)  2,485,924,80 


Note:  "*"  indicates  the  cost  has  been  amortized  over  an  8-year  period. 
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Table  6 


Annual  Cost  of  PACS  at  WRAMC  (Years  2-8) 

Year 

Equipment 

Maintenance  &  Engineer 

Storage 

Total  Annual  Cost  ($) 

2 

844,505.88* 

1,923,499.00 

6,720.00* 

2,774,724.80 

3 

844,505.88* 

1,732,375.00 

6,720.00* 

2,583,600.80 

4 

844,505.88* 

1,673,444.00 

6,720.00* 

2,524,669.80 

5 

844,505.88* 

1,240,249.00 

6,720.00* 

2,091,474.80 

6 

844,505.88* 

1,148,841.00 

6,720.00* 

2,000,066.80 

7 

844,505.88* 

961,636.00 

6,720.00* 

1,812,861.80 

8 

844,505.88* 

961,636.00 

6,720.00* 

1,812,861.80 

Note:  indicates  the  cost  has  been  amortized  over  an  8-year  period. 
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Table  7 

Annual  Cost  of  Film-Based  Radiology  at  WRAMC _ 

Item  Quantity  Individual  Cost  ($)  Total  Cost  ($) 


X-Ray  Apparatus  (no  new  equipment  proposed)  0 

Developers  (no  new  equipment  proposed)  0 

Yiewboxes  (no  new  equipment  proposed)  0 


Film 

463,578.57 

Chemicals 

Contract  for  Silver  Recovery  and  Hazardous  Waste  Disposal 

17,360.00 

14,000.00 

Film  Jackets 

Master  26,840 

Report  Jackets  23,240 

T  eaching  Files  1,130 

Inserts  122,000 

$42  per  100 
$38  per  100 
$38  per  100 
$70  per  500 

11,272.80 

8831.20 

429.40 

17,080.00 

Maintenance  (Annual  contract  to  clean  processors) 

15,600.00 

Total  Annual  Supply  and  Production  Cost 

548,151.97 

Personnel 

File  Room  Clerk  (GS-4)  7  (diagnostic) 

Darkroom  Technician  (GS-4)  1 

File  Room  Clerk  (GS-4)  1  (therapy) 

19,146 

19,146 

19,146 

134,022.00 

19,146.00 

19,146.00 

Total  Personnel  Cost 

Total  Annual  Supply,  Production  and  Personnel  Cost 

172,314.00 

720,465.97 

Space  for  Archive 

Diagositic 

Radiation  Therapy 

Nuclear  Medicine 

Orthopaedics 

Total  Square  Footage 

TOTAL  ANNUAL  STORAGE  COST  (@  $140  per  square  foot) 

4366 

36 

224 

252 

4878 

193,200.00 

TOTAL  ANNUAL  COST  OF  FILM-BASED  RADIOLOGY 


913,665.97 
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Table  8 

Demographics  of  Survey  Participants 

Number  Percentage  (Rounded) 


Position 

Medical  Student  10  8 

Intern  18  15 

Resident  39  33 

Attending  12  10 

Staff  24  20 

Other  16  14 

Gender 

Male  99  83 

Female  17  14 

Unknown  (unmarked)  3  3 

Status 

Active  Duty  109  92 

Contract  2  2 

Other  7  6 

1  <1 


Unknown  (unmarked) 
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Table  9 

Descriptive  Statistics  for  All  Continuous  Variables  (n=l  19) _ 

Question  #  Mean  Standard  Deviation  Minimum  Maximum 


1 

1.47 

.66 

2 

2.71 

1.09 

3 

2.21 

.96 

4 

2.02 

.85 

5 

2.03 

.97 

6 

1.76 

.96 

7 

2.22 

.93 

8 

2.30 

1.02 

9 

2.21 

.81 

10 

1.98 

.65 

11 

2.17 

.87 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 


4 

5 

4 

5 
5 
5 
5 
5 
5 
5 


1 


5 
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Table  10 

Overall  Satisfaction  (Question  1 1)  By  Department 


Department 

Mean 

Standard  Deviation 

Minimum 

Maximum 

Radiology 

2.07 

.73 

1 

4 

Emergency 

1.58 

.51 

1 

2 

Medicine 

1.70 

.53 

1 

3 

Surgical  Intensive  Care 

1.95 

.65 

1 

4 

Cardiothoracic  Surgery 

2.69 

.95 

2 

5 

Pulmonary 

2.69 

1.01 

2 

5 

Urology 

3.00 

.78 

2 

4 
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Table  11 

Overall  Satisfaction  by  Type  of  Healthcare  Provider _ 

Type  of  Provider  Mean  Standard  Deviation  Minimum  Maximum 


Student 

2.03 

.77 

Staff 

2.44 

.98 

1 


5 
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Table  12 

Correlations  Between  Mean  Scores  of  Provider  Responses  to  Questions  1-10  and  Overall 

Satisfaction  (Question  11) _ 

Q1  Q2  Q3  Q4  Q5  Q6  Q7  Q8  Q9  Q10  Qll 

1.  Q1  1.000  .095  .293**  .452**  .457**  .135  .327**  .252**  .385**  .314**  .399** 

2. Q2  .095  1.000  .111  .051  .007  .129  -.196  -.028  -.042  -.055  -.049 

3.  Q3  .293**  .111  1.000  .467**  .284**  .298**  .259**  .287**  .288**  .330**  .261** 

4. Q4  .452**  .051  .467**  1.000  .551**  .333**  .539**  .225*  .156  .349**  .433** 

5.  Q5  .457**  .007  .284**  .551**  1.000  .333**  .369**  .317**  .176  .363**  .513** 

6. Q6  .135  .129  .298**  .333**  .333**  1.000  .228**  .107  .086  .372**  .245** 

7.  Q7  .327**  -.196*  .259**  .539**  .369**  .228*  1.000  .188*  .263**  .524**  .355** 

8. Q8  .252**  -.028  .287**  .225**  .317**  .107  .188*  1.000  .487**  .327**  .585** 

9. Q9  .385** -.042  .288**  .156  .176  .086  .263**  .487**  1.000  .504**  .505** 

10.  Q10  .314**  -.055  .330**  .349**  .363**  .372**  .524**  .327**  .504**  1.000  .537** 

11.  Qll  .399**  -.049  .261**  .433**  .513**  .245**  .355**  .585**  .505**  .537**  1.000 

Note.  "**"  indicates  that  the  correlation  is  significant  at  the  0.01  level  (2-tailed).  "*"  indicates 
that  the  correlation  is  significant  at  the  0.05  level  (2-tailed). 
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Table  13 


Stepwise  Multiple  Regression  Analysis 


Model 

B 

SE  B 

P 

t 

Sig. 

1.  (Constant) 

1.032 

.160 

6.440 

.000 

Q8 

.496 

.064 

.585 

7.798 

.000 

2.  (Constant) 

.257 

.202 

1.269 

.207 

Q8 

.389 

.060 

.458 

6.442 

.000 

Q10 

.515 

.095 

.387 

5.439 

.000 

3.  (Constant) 

8.925E-02 

.195 

.457 

.649 

Q8 

.337 

.058 

.397 

5.769 

.000 

Q10 

.409 

.093 

.307 

4.380 

.000 

Q5 

.246 

.062 

.275 

3.942 

.000 

4.  (Constant) 

-3.231E-02 

.200 

-.162 

.872 

Q8 

.280 

.063 

.330 

4.454 

.000 

Q10 

.313 

.102 

.325 

3.076 

.003 

Q5 

.261 

.062 

.292 

4.226 

.000 

Q9 

.187 

.084 

.175 

2.210 

.029 
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Table  14 

Model  Summary 

Model 

r 

R2 

Std.  Error  of  the  Estimate 

1. 

.585 

.342 

.71 

2. 

.690 

.476 

.63 

3. 

.734 

.538 

.60 

4. 

.746 

.557 

.59 

L 
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Table  15 


ANOVA 


Model 

Sum  of  Squares 

df 

Mean  Square 

F 

Sig 

1.  Regression 

30.325 

1 

30.325 

60.812 

<.0001 

Residual 

58.344 

117 

.499 

Total 

88.669 

118 

2.  Regression 

42.179 

2 

21.090 

52.622 

<0001 

Residual 

46.490 

116 

.401 

Total 

88.669 

118 

3.  Regression 

47.715 

3 

15.905 

44.661 

<0001 

Residual 

40.955 

115 

.356 

Total 

88.669 

118 

4.  Regression 

49.397 

4 

12.349 

35.847 

<0001 

Residual 

39.273 

114 

.344 

Total 

88.669 

118 
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Table  16 

Additional  Survey  Questions 

Number 


Desire  Dedicated  Review  Station  in  Clinic 
Yes  103 

No  4 

No  Opinion  Marked  12 

Desire  reports/images  displayed  on  personal  computer 
Yes  109 

No  8 


Percentage  (Rounded) 


87 

3 

10 

92 

7 


No  Opinion  Marked 


2 


2 
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Table  17 

Provider  Comments 


Emergency  Room 

1 .  My  only  complaint  is  that  frequently  old  (archived)  x-rays  are  not  accessible  in  the  system  for 
comparison. 

Cardiothoracic  Surgery 

1 .  System  works  very  well,  just  need  terminals  at  each  clinic 

2.  Would  like  to  see  more  PACS  terminals,  especially  in  general  surgery/thoracic  clinic. 

3.  Lack  of  viewbox  in  clinic  is  an  enormous  degradation  in  the  quality  of  my  practice. 

4.  Need  to  have  the  system  in  the  CT  clinic  for  accessibility. 

5.  Need  terminals  in  the  general  surgery  clinic  where  we  see  our  patients. 

6.  Need  PACS  in  the  clinic.  Currently  must  go  two  floors  to  review  films. 

7.  Need  more  viewing  stations  at  our  surgery  clinic. 

8.  Overall  a  good  system,  but  needs  wider  implementation. 

9.  Limited  input  due  to  limited  use  of  PACS. 

Urology 

1. 1  have  used  this  system  elsewhere,  but  not  at  WRAMC. 

2.  Not  enough  viewing  stations. 

3.  All  my  "disagrees"  related  to  the  fact  that  there  is  no  imager  available  in  my  area. 

Pulmonary 

1.  Drawback  is  the  inability  to  simultaneously  view  multiple  films. 

2.  Too  few  reading  stations;  films  not  always  on  the  system. 

3. 1  have  not  used  PACS. 

Surgical  Intensive  Care  Unit 

1 .  There  is  a  significant  problem  with  retrieving  old  (archived)  films;  is  great  for  new  films  -  a 
large  improvement. 

Department  of  Medicine 

1 .  Only  problem  is  that  images  get  archived  too  soon. 

2.  Reports  would  be  great  to  have  displayed  for  every  image  with  voice  writer.  Speaker  talks  too 
fast  to  get  all  info  quickly. 

3. 1  would  absolutely  like  to  have  reports  and  images  displayed  on  my  personal  computer. 
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Figure  1.  Patient  Terminal  (left)  and  Computer  Radiology  Acquisition  Workstation 


(right) 
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Figure  3.  A  “4A”  Workstation  for  use  by  radiologists. 
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Figure  4.  A  “2C”  Viewing  Station 
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Survey  Instrument 
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PICTURE  ARCHIVE  AND  COMMUNICATION  SYSTEMS 
(PACS) 

In  partial  fulfill  ment  of  master’s  degree  work,  I  am  investigating  the  impact  of  the  PACS  on  clinical  practice  at  Walter  Reed  Army  Medical 
Center  (WRAMC).  Would  you  please  take  a  few  moments  to  answer  the  following  questions?  Your  ratings  will  provide  valuable  feedback. 
Although  I  ask  for  some  background  information,  please  be  assured  that  the  survey  is  anonymous  and  confidential.  Thank  you  for  your 
assistance. 

Hunter  R.  Clouse,  COL,  DC 

Please  rate  how  much  you  agree  or  disagree  with  the  following  items  on  the  scale  provided.  Circle  the  appropriate  number  corresponding  with 
your  response.  For  example.  Strongly  Agree  =1,  Agree  Somewhat  =  2,  No  Opinion/Neutral  =3,  Disagree  =  4,  Strongly  Disagree  =  5 


Strongly  Agree 

Agree 

No  Opinion 

Disagree 

Strongly  Disagree 

1.  PACS  provides  rapid  access 
to  radiographic  images.  1 

2 

3 

4 

5 

2.  PACS  provides  rapid  access 

to  the  radiologist's  formal  report.  I 

2 

3 

4 

5 

3.  PACS  results  in  fewer 
duplicated  (repeated) 

radiographic  examinations.  1 

2 

3 

4 

5 

4.  PACS  improves  the  quality 

of  care  delivered.  1 

2 

3 

4 

5 

5.  PACS  increases  the  efficiency 

of  my  clinical  practice.  1 

2 

3 

4 

5 

6.  The  ability  to  review  images 
simultaneously  in  more  than  one 
location  (i.e.  the  presence  of  multiple 
viewing  stations)  is  important  in  my 
clinical  practice.  1 

2 

3 

4 

5 

7.  The  diagnostic  accuracy  (technical 
quality)  of  digital  images  is  equal  to  or 
exceeds  that  of  traditional  film-based 
techniques.  1 

2 

3 

4 

5 

8.  I  find  the  PACS  at  WRAMC 

to  be  “user  friendly.”  1 

2 

3 

4 

5 

9.  The  PACS  at  WRAMC  is 

reliable.  1 

2 

3 

4 

5 

10.  Iam  confident  in  my  diagnoses 
using  digital  images.  1 

2 

3 

4 

5 

1 1 .  Overall,  I  am  satisfied  with  the 
performance  of  the  PACS  at  WRAMC.  1 

2 

3 

4 

5 

1 .  What  is  your  position?  Medical  Student  Intern  Resident  Attending  Staff  Other  (Specify): 


What  is  your  gender? 

Male 

Female 

What  is  your  status? 

Active  Duty 

Contract 

Other  (Specify): 

4.  Would  you  like  to  have  a  dedicated  review  station  located  in  your  clinic?  (Please  check  one)  Yes  ___  No _ 

Would  you  like  to  see  reports  and  images  displayed  on  your  personal  computer?  (Please  check  one)  Yes  ___  No 


5. 
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Instrument  to  Establish  Content  Validity 
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PICTURE  ARCHIVE  AND  COMMUNICATION  SYSTEMS 
(PACS) 

Please  rate  the  relevance  of  each  of  the  following  to  the  listed  objective.  Circle  the  appropriate  number  corresponding  with  your 
response.  For  example,  Not  Relevant  =1,  Somewhat  Relevant  =  2,  Quite  Relevant  =3,  Very  Relevant  =  4. 

Not  Relevant  Somewhat  Relevant  Quite  Relevant  Very  Relevant 

BENEFITS  TO  THE  DIAGNOSTICIAN  (RADIOLOGIST) 

1 .  PACS  provides  rapid  access 

to  radiographic  images.  12  3  4 

2.  The  diagnostic  accuracy  (technical 
quality)  of  digital  images  is  equal  to  or 
exceeds  that  of  traditional  film-based 

techniques.  12  3  4 

3 .  Iam  confident  in  my  diagnoses 

using  digital  images.  12  3  4 


BENEFITS  TO  THE  REFERRING  PHYSICIAN 

4.  PACS  increases  the  efficiency 

of  my  clinical  practice.  12  3  4 


5.  The  ability  to  review  images 

simultaneously  in  more  than  one 
location  (i.e.  the  presence  of  multiple 
viewing  stations)  is  important  in  my 
clinical  practice.  1 

6.  I  find  the  PACS  at  WRAMC 

to  be  “user  friendly.”  1 


2  3  4 

2  3  4 


7.  The  PACS  at  WRAMC  is 

reliable.  12  3  4 


8.  Overall,  I  am  satisfied  with  the 

performance  of  the  PACS  at  WRAMC.  12  3  4 


9.  PACS  provides  rapid  access 

to  the  radiologist’s  formal  report.  12  3  4 


BENEFITS  TO  THE  PATIENT 

10.  PACS  results  in  fewer 
duplicated  (repeated) 

radiographic  examinations  12  3  4 

1 1 .  PACS  improves  the  quality 

of  care  delivered.  12  3  4 
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